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Abstract 

Background: Population-based epidemiological studies of essential and other tremors have need of a rapid yet accurate means to assess tremor, especially mild 
tremors. Handwriting is often afTected by tremor, and a hand-drawn spiral can provide investigators with objective rather than self-reported data. We present a 
semi-quantitative, ordinal scale to rate hand-drawn spirals. The scale, which includes values for mild tremor, is accompanied by photographic examples of spirals of 
each rating, providing a visual template for guidance. 

Methods: This study, conducted within the framework of a population-based epidemiological study of 5,000 individuals aged 60 and older in Shanghai, asked 
enrollees to draw an Archimedes spiral with each hand. Spirals were rated using an ordinal scale (0, 0.5, 1, 1.5, 2, and 3). Three raters rated an initial set of 548 
spirals. Four raters rated a subsequent set of 200 spirals using a visual template for guidance. 

Results: Initial agreement (548 spirals) was good (r values ranged from 0.49 to 0.62, all p<0.001). Subsequent agreement (200 spirals and using visual template) 
improved (r values ranged from 0.67 to —0.91, all p<0.001). 

Discussion: This tool will be useful to researchers who are attempting to rapidly assess action tremor in their field surveys. 
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Introduction 

Population-based epidemiological studies of the prevalence, 
incidence, and determinants of a variety of types of tremor, 
including essential tremor (ET), 1-3 toxin-induced tremors, 4-8 and 
enhanced physiological tremor/ require a rapid yet accurate means 
to assess tremor. The assessment of action tremor in such settings 
poses practical challenges. For example, although a neurologist 
specializing in movement disorders is best positioned to provide the 
most valid measure of tremor severity, it is often not feasible for 
such a person to examine thousands of participants in field settings. 
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Portable tools for the assessment of tremor, such as the digitizing 
tablet, 10 are available, but these have not been validated in such 
field studies. 

Handwriting is often affected by tremor, 1 1 and a hand-drawn spiral 
can provide investigators with objective rather than subjective (i.e., 
self-reported) data. Spirals can be collected in the field and rapidly 
rated at a later point in time. 

A critical issue is that population-based studies often involve the 
evaluation of individuals whose tremor is on the mild end of the 
spectrum. 12 Hence, standard rating scales (e.g., 0, 1, 2, 3), used in 
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clinical settings to assess more severe tremor, may not be of value. We 
have devised a semi-quantitative scale for rating spirals: the scale more 
precisely distinguishes between tremors in the mild end of the 
spectrum (e.g., ratings of 0, 0.5, 1, 1.5), which is in marked contrast 
to previous scales we have used that did not do so. 

In this report, we present the scale and describe our methods to 
improve upon the application of the scale so as to maximize its 
reliability. The issue of reliability is important, that is, whether 
different raters are able to use the scale to derive comparable ratings. 
Indeed, demonstrating reliability is an important initial step in 
epidemiological research as reliability may be the only measure of 
data quality in situations in which validity is difficult to assess. 1 
Importantly, the scale we describe is accompanied by photographic 
examples of spirals of each rating, providing a visual template for 
guidance, and, as we show in this report, the use of the visual template 
improves upon reliability. We hope this tool will be of use to 
researchers who are attempting to rapidly assess tremor in their field 
surveys and populations. 

Methods 

Study setting 

This study was conducted as part of a population-based epidemio- 
logical study, the Prevalence Study of Alzheimer's Disease and Mild 
Cognitive Impairment in Shanghai, China, which began in October 
2009. The study, which sampled 5,000 individuals aged 60 and older 
in an urban community, focuses on cognitive impairment and 
dementia, although several other neurological disorders, including 
ET, will be assessed. At the time of enrollment, written informed 
consent was obtained from all participants and/or their legally 
acceptable representative. The study was approved by the ethics 
committee of Huashan Hospital, Fudan University, Shanghai. 

Spiral drawings 

Enrolled participants were asked to draw an Archimedes spiral 
with each hand. Spirals were drawn on a standard 8.5 x 11 inch 
sheet of paper using a pen while the participant was seated at a 
table. The paper was centered at right angles directly in front of 
them and held down by their other hand. The drawing hand was 
not allowed to rest or be supported when the spiral was being 
drawn. Participants started at the center of the page, without lifting 
their pen. 1 

Spiral rating: In-person training session 

Two investigators (a neurologist, Q.Z., and a neuroepidemiologist, 
D.D.) underwent a 2-hour, in-person training session in New York 
with a senior movement disorder neurologist with expertise in tremor 
evaluation (E.D.L.). During the training session, the three individuals 
co-reviewed 300 hand-drawn spirals using the rating scale described 
below. The 300 spirals ranged in severity from normal (no tremor) to 
marked tremor. 
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Spiral rating: Initial independent ratings 

Two months later, the three investigators, blinded to clinical 
information, independently rated spirals that were available from the 
first 548 enrollees. The rating scale was as follows: 

0 = Absolutely no oscillations anywhere. 

0.5 = Subtle, low-amplitude oscillations are present in a few spots. 
Oscillations are not consistently present throughout the spiral. 
1.0 = Low-amplitude oscillations are present in multiple places. 
Examining at least one of the spiral's quadrants reveals the presence of 
these low-amplitude oscillations that occur in each larger and larger 
line of the spiral within that quadrant. 

1.5 = Low-amplitude oscillations are present in multiple places and 
oscillations can at times reach moderate amplitude. 

2 = Moderate-amplitude oscillations that are present in many areas of 
the spiral. 

3 = Oscillations reach large amplitude in one or more places. Lines 
may overlap. Pen may lift off the paper. 

On each spiral, the raters were careful to distinguish clear, regular 
oscillations from sloppiness, spatial errors, and other irregularities or 
movement disfluencies that were not strictiy oscillatory. 

Spiral rating: Second independent ratings using visual templates 

To further improve on the inter-rater reliability, the senior 
movement disorder neurologist carefully assembled visual examples 
of spirals that he had rated, including two examples each of the spirals 
rated as 0.5, 1.0, 1.5, 2, and 3 (Figures 1-5). These spirals were 
electronically scanned to produce a visual template to be used as a 
reference guide during the rating process. The investigative team in 
China was asked to assemble 200 spirals with a range of scores (0-3) so 
that they could be independently rated by the movement disorder 
neurologist, the two previous investigators on the Chinese team, and a 
new investigator (a second neurologist) on the Chinese team (H.M.). 
The new investigator did not have the benefit of the initial in-person 
training session in New York or experience with the prior 548 rated 
spirals. These 200 spirals were independently rated by the four 
investigators. 

Statistical analyses 

Statistical analyses were performed using SPSS (Version 18.0). 
Agreement between raters in the initial set of 548 spirals and in the 
subsequent set of 200 spirals was assessed using Spearman's correlation 
coefficient (r) and intraclass correlation coefficients (ICCs). For ICCs, 
absolute agreement was assessed rather than consistency. 

Results 

In the initial set of 548 spirals, the US neurologist assigned the 
following ratings: 0 (n = 5), 0.5 (n = 280), 1 (n = 208), 1.5 (n = 35), 2 
(n = 17) and 3 (n = 3). The agreement between raters ranged from 
r = 0.49 (p<0.001) to r = 0.62 (p<0.001), indicating good agreement 
(Table 1). ICC values tended to be slighdy higher than r values (Table 1). 
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In the subsequent set of 200 spirals, the US neurologist assigned the 
following ratings: 0 (n = 4), 0.5 (n = 20), 1 (n = 84), 1.5 (n = 48), 2 
(n = 38) and 3 (n = 6). The agreement between raters ranged from 
r = 0.67 (p <0.001) to r = 0.91 (p<0.001), indicating good agreement 
(Table 2). ICC values tended to be slighdy higher than r values (Table 2). 
The agreement between the raters improved when compared with the 
initial agreement (548 spirals). Thus, for the agreement between the US 
neurologist and the Chinese neurologist 1, the r value increased from 0.56 
to 0.74; for the agreement between the US neurologist and the Chinese 
neuroepidemiologist, the r value increased from 0.49 to 0.73; and for the 
agreement between the Chinese neurologist 1 and the Chinese 
neuroepidemiologist, the r value increased from 0.62 to 0.91 (Tables 1 
and 2). Chinese neurologist 2, who had received no in-person training and 
who only used the visual templates, had high agreement with the other 
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raters (r = 0.67 with the US neurologist; r = 0.78 with Chinese 
neurologist 1; and r = 0.87 with the Chinese neuroepidemiologist). 

We assessed whether the agreement between raters differed with 
respect to the rating of spirals that were drawn with the right or left 
hand. Agreement was slightly higher for the left hand; for example, for 
the agreement between the US neurologist and Chinese neurologist 1 , 
r (right) = 0.70 and r (left) = 0.78; for the agreement between the US 
neurologist and Chinese neurologist 2, r (right) = 0.60 and r (left) = 
0.78; and between the two Chinese neurologists, r (right) = 0.74 and r 
(left) = 0.87. We also stratified ratings into low (i.e., the US neurologist 
assigned ratings of 0 or 0.5) vs. high (the US neurologist assigned 
ratings of 2 or 3) to see whether the severity of tremor affected the level 
of agreement. We did not find that agreement differed across these two 
strata. 
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Figure 3. (A, B) Two Spirals that Received Ratings of 1.5. 




Discussion 

Movement disorder neurologists cannot practicably be sent into the 
field to personally examine thousands of study subjects nor is it 
practicable to videotape thousands of neurological examinations in the 
field for later viewing. Hence, a screening procedure is necessary. The 
problem is that screening questionnaires for ET lack sensitivity, 
particularly for mild tremor. 1 ' 1 As an alternative, handwriting samples 
allow for the rapid collection of objective rather than self-reported 
data. Furthermore, for other types of tremor (e.g., toxin-induced 
tremors) the value of screening questionnaires is not known; empiric 
data on the presence and severity of tremor are of greater value. We 
are aware of one other example of a rating scale for action tremor that 
provides visual examples; however, a problem with that scale is that 
ratings are from 0 to 10, and this large number of scale steps has the 
potential to produce discrepancies in rater agreement. 16 



In the current study, we demonstrated that the semi-quantitative 
rating scale we use is reliable. Indeed, the second Chinese neurologist, 
who had had no in-person training and who only used the visual 
templates, demonstrated a high agreement with the other raters 
(r = 0.67 with the US neurologist; r = 0.78 with Chinese neurologist 
1; and r = 0.87 with the Chinese neuroepidemiologist). The 
photographic examples of spirals of each rating provide an easy to 
use visual template for guidance, and this improves reliability. 

This visual template-based method can be used in a variety of field 
settings. For example, it is often important to decide in the field 
whether to incorporate a more detailed assessment after an initial 
screening evaluation, and that decision needs to be made on the spot. 
Thus a screening spiral that receives a rating above a certain 
predetermined threshold (e.g., 1.5) might be an entree to a second and 
more detailed diagnostic neurological examination on the same day of 
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Figure 5. (A, B) Two Spirals that Received Ratings of 3. 



Table 1 . Inter-rater Agreement in Ratings of Tremor in the First Set of 548 Spirals 



US Rater 



Neurologist I (China) Neuroepidemiologist (China) 



US Rater 



r = 0.56, p<0.00l 
ICC = 0.68, p<0.00l 



Neurologist I (China 




Neuroepidemiologist (China) 

Abbreviations: ICC, intraclass correlation coefficient; r, Spearman's correlation coefficient. 



r = 0.49, p<0.00l 
ICC = 0.49, p<0.00l 
r = 0.62, p<0.00 1 
ICC = 0.67, p<0.00l 




Table 2. Inter-rater Agreement in Ratings of Tremor in Second Set of 200 Spirals 



US Rater 


Neurologist 1 (China) 


Neurologist 2 (China) 


Neuroepidemiologist (China) 


US rater 


r = 0.74, p<0.00l 
ICC = 0.79, p<0.00l 


r = 0.67, p<0.00l 
ICC = 0.70, p<0.00l 


r = 0.73, p<0.00l 
ICC = 0.77, p<0.00l 


Neurologist 1 (China) 




r = 0.78, p<0.00l 
ICC = 0.81, p<0.00l 


r = 0.91, p<0.00l 
ICC = 0.92, p<0.00l 


Neurologist 2 (China) 






r = 0.87, p<0.00l 
ICC = 0.89, p<0.00l 


Neuroepidemiologist (China) 


Abbreviations: ICC, intraclass correlation coefficient; r 


Spearman's correlation coefficient. 
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testing. The other important issue is that the use of these templates will 
ensure that field workers have calibrated their ratings against the visual 
examples of scores assigned by a senior movement disorder specialist 
with expertise in tremor. The initial screen is critical; if inaccurate, 
cases will either be under-ascertained or alternatively too many cases 
will be referred for a second, more-detailed evaluation, placing an 
undue burden on study resources. 

We recognize that this study had limitations. First, it is important to 
recognize that these spirals assess action tremor but they do not allow 
one to definitively distinguish parkinsonian or dystonic action tremor 
from ET. This being said, the presence of micrographia or the absence 
of a consistent spiral axis 17 would argue in favor of parkinsonism and 
dystonia. Also, if handwriting is not affected by tremor, then this 
spirography method will under-ascertain tremor cases. As the focus of 
this paper was on inter-rater agreement, we did not present data using 
Cronbach's alpha, which is a measure of the internal consistency of 
data. The current analyses used pen and paper to capture spiral data. 
Whether similar field data could be captured on a computer or a 
digitizing tablet and then rated reliably remains to be determined. 

It is our hope that the tool we describe here will be of use to 
researchers who are attempting to rapidly screen for tremor in their 
field surveys and population-based studies, and will pave the way for 
population-based studies of tremor that utilize an objective but not 
burdensome initial screening process. 
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